In order to resolve this problem, we need some kind of transformation of the \(p_i\) values that will cause them to be contained within the interval from zero to one. It turns out that the transformation we are looking for is the logit transformation.

The logit transformation converts a probability into the log-odds. Formally,

\[logit(p)=log(\frac{p}{1-p})\]

There are really two parts to this transformation. First, we convert from probabilities to odds by taking \(p/(1-p)\). We have seen odds before in this course, in the section on two-way tables. There we learned how to calculate the odds ratio. The odds is the ratio of the expected number of successes to failures. So, if \(p=0.75\), we expect that three out of every four trials will produce successes, on average. In terms of the odds, we expect three successes for every one failure. To convert:

\[O=\frac{p}{1-p}=\frac{0.75}{1-0.75}=\frac{0.75}{0.25}=3\]

Why do we convert from probabilities to odds? The advantage of the odds is that it has no upper limit. As the probability gets closer and closer to onem, the odds will approach infinity, with no finite limit. Therefore, any non-negative number for the odds can be converted back into a probability that will give sensible values between zero and one. I can convert back to a probability by:

\[p=\frac{O}{1+O}\]

So, for the case of \(O=3\) above:

\[p=\frac{3}{1+3}=\frac{3}{4}=0.75\]

Lets choose a really high odds like \(O=100,000\). If we convert back to a probability:

\[p=\frac{100000}{1+100000}=\frac{1}{100001}=0.99999\]

We get a very high probability, but its still less than one.

This partially helps us with our problems. If we were to look at a linear relationship between the odds of success and our independent variables we would get sensible probabilities no matter how high the predicted odds. However, it only partially helps us because it would still be possible to get negative odds from such a linear model which would be nonsensical.

The second step of logging the odds will get us all the way there. If I log a value below one, I will get a negative value and that logged value will approach negative infinity as the original value approaches zero. So a log-odds can always be converted back to a probability that will lie between zero and one.

To convert from a log-odds \(g\) to a probability, I take:

\[p=\frac{e^g}{1+e^g}\]

The value \(e^g\) converts from log-odds to odds and then I just use the formula for converting from an odds to a probability.

In essence the log-odds, or logit, transformation stretches out my probabilities across the entire number line. Lets see what this looks like graphically for a sequence of probabilities from 1% to 99%: